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uring the last decade many states have tried 
to devise more robust and coherent 
instructional policies, in efforts to make 
teaching and learning more thoughtful and 
demanding. Policymakers and reformers have pressed 
teachers to help students understand mathematical concepts, 
to interpret serious literature, to write creatively about their 
own ideas and experiences, and to converse thoughtfully 
about history and social science. But these efforts to reform 
instruction have encountered skepticism about the link 
between policy and pedagogy. Skeptics ask if it is 
reasonable to expect state policies to steer teaching and 
learning sharply away from long-established conventional 
practice, noting that previous efforts to change practice on 
a large scale have failed. 



As instructional policy has moved to the top of many state 
education agendas in the past ten to fifteen years, interest 
in the relations between policy and practice has grown. In 
this issue of CPRE Policy Briefs, we report encouraging 
findings from an important study that addresses these 
relationships. We use data from a 1 994 survey of California 
elementary school teachers to probe the classroom effects 
of state efforts to reform mathematics teaching and learning 
in California. We report that policy changes did lead both to 
changed classroom practice and to improved student 
performance. 
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In this brief, we develop a rudimentary model of the 
relationship between policy and practice. Student 
achievement is the ultimate dependent measure; teachers’ 
reported classroom practice in mathematics is an influence 
on achievement, but practice also is a measure of the effects 
of teachers’ learning opportunities about new math 
curriculum. We present results which show that teachers’ 
learning opportunities influenced their practice, and that both 
teachers’ learning opportunities and their practice influenced 
students’ mathematics achievement. The results suggest 
that teachers’ practice can change in ways that favorably 
influence student achievement, and that policy can play an 
important role in making those changes possible. 

We begin with a review of the California reform, briefly 
describe the research approach, and then discuss the major 
findings. 

S(Dlh®@l Rcgfoinnn) m (Mofomoa 

Like other states in the 1980s, California sought to improve 
student achievement by redesigning state policies and other 
mechanisms specific to instructional policy, including 
student curriculum, assessments, and professional 
development. The goal was to change classroom practice 
across the state, and thus improve student performance. 
These effects would depend in considerable part on 
professional learning, for the policies could not change 
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practice unless teachers learned new 
views of mathematics. Teachers’ 
learning opportunities thus were a key 
policy instrument. 

The California Department of 
Education took the first step in 1985 
when it issued a new Mathematics 
Framework; the endeavor continues 
today, though much modified. The 
1985 Mathematics Framework called 
for more intellectually ambitious 
instruction, for more engaging 
mathematics work for students, for 
teachers to open discourse about math 
in their classrooms and to pay more 
attention to students’ mathematical 
ideas, and for teachers to help students 
understand math instead of just 
memorizing facts and operations. 
Although officially only advisory to 
local districts, the Mathematics 
Framework was a central part of state 
instructional policy. 

Shortly after issuing the new 
Mathematics Framework, the 
California State Board of Education 
attempted to use textbook adoption as 
a policy instrument. State approval 
carries great weight with local districts 
because they receive state aid for using 
approved texts. 1 The State Board 
initially rejected most math textbooks 
because they did not conform to the 
new Mathematics Framework. After 
much debate, partial revision of some 
texts, and political conflict, the State 



Board approved most of them. Unhappy 
with the limited changes in texts, state 
officials decided that textbook revision 
might not be the best means to 
encourage reform. 

State officials began to stimulate the 
development of other curriculum 
materials — especially topic-centered 
teaching modules called “replacement 
units.” These modules would support 
changed mathematics teaching, each 
focusing on a few weeks of classwork. 
State Department of Education staffers 
also encouraged teacher professional 
development centered on the reforms. 
For the new instructional ideas to be 
taken seriously, teachers and other 
educators would have to learn a great 
deal. Moreover, the Mathematics 
Framework offered only general 
guidance to teachers, so its effective 
use would heavily depend on teachers’ 
knowledgeable use of the guidance. 

The California Department of 
Education also used its student 
assessment system as another means 
to change teaching practice, and 
devoted considerable effort to revising 
the tests and aligning them with the 
Mathematics Framework. Some 
reformers were uneasy about testing, 
but others assumed that new tests 
could help. Once the state began 
testing students on the new 
mathematical content and methods, 
they reasoned, test scores would drop 



because the material would be 
unfamiliar and more difficult. Teachers 
and the public would notice the lower 
scores, which would generate pressure 
for better results. Teachers would 
respond to the pressure by paying 
attention to the new tests and changing 
instruction. 

The California Department of 
Education had difficulty revising the 
tests in part because it was a 
formidable task, in part because of 
disagreements between the State 
Superintendent of Public Instruction 
and the Governor, and in part because 
of the Superintendent’s legal 
difficulties and ultimate departure. The 
revisions were finally completed and 
the new tests, the California Learning 
Assessment System (CLAS), were 
administered in 1993 and 1994. Just 
as state education leaders had 
predicted, scores were lower and the 
public took note. But a storm of protest 
erupted after the 1 993 test results were 
published: scores were generally low, 
and a technical panel gave low grades 
to features of the assessment and its 
administration. Some modifications 
were made for the 1 994 administration 
of the CLAS tests, partly in response 
to the outcry over low scores, but it 
was too late. The entire enterprise came 
under question. Some criticized the new 
tests because, they claimed, they paid 
little attention to the “basics” and instead 
encouraged “critical thinking” or 
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“outcomes-based education.” Others 
raised questions about the technical 
quality of the test, its administration, 
reporting, and analysis, especially the 
“subjectivity” of items and scoring. The 
Governor, who had wanted a testing 
program that produced individual 
student scores rather than school 
average scores, canceled the testing 
program. 

California’s instructional reforms 
continue, but no test linked to the 
Mathematics Framework now exists, 
and the Framework has just been 
revised amidst continuing controversy 
about “critical thinking” and the 
“basics.” 




The research reported here began 
roughly a decade ago with a research 
group at Michigan State University. 
The researchers wanted to learn how 
teachers responded to the wave of 
ambitious state instructional policies 
in California and several other states, 
and to use this knowledge to improve 
understanding of policymaking and 
implementation. Members of the 
research group studied documents, 
visited elementary schools and 
classrooms in three California school 
districts, and followed the same 
schools and teachers for four or five 
years. 2 The researchers also followed 
developments in state and district 
offices by interviewing many state and 
district administrators and reformers, 
and by studying efforts to improve 
teacher knowledge and skills in various 
professional development projects. 

As the researchers studied classrooms 
and mathematics teaching in 
California, they saw that the reforms 
required extensive learning. The reforms 



could not be enacted unless educators, 
parents, and policymakers revised much 
of their knowledge of both mathematics 
and instruction, and unless educators 
developed new ways to teach the 
subject. Hence, implementation of these 
reforms would require learning 
opportunities that did not exist in 1985. 3 
California’s mathematics instructional 
policy entailed a re-education program 
for teachers. 

If implementation of the new 
Mathematics Framework was partly a 
matter of teacher learning, and if most 
teachers could not teach themselves 
what they did not know, then some 
agency or agencies would have to teach 
the teachers. That led the Michigan 
State research team to explore several 
issues: What learning opportunities 
were available to teachers and others 
responsible for implementing the new 
Mathematics Framework? What 
mathematics content and pedagogies 
were teachers taught? Was there any 
connection between teachers’ learning 
opportunities and practice? 

To answer these and other questions, 
the research team supplemented the in- 
depth studies of California classrooms 
and teachers with a one-time survey 
of 1 ,000 teachers, sampled to represent 
the population of second through fifth 
grade elementary school teachers in 
California. 4 Questions included 
teachers’ opportunities to learn about 
the mathematics reforms, about their 
mathematics teaching practice and its 
relation to some of the dimensions 
advocated by the new Framework, and 
how their practice followed or varied 
from conventional practice. 
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Teachers reported three different types 
of learning opportunities: study of 
specific math curriculum materials 
that had been created to advance the 
reforms; study of special topics and 
issues related to the mathematics 
reform; and participation in reform 
networks and activities. Table 1 
presents evidence of the time teachers 
spent in different types of learning 
opportunities. Teachers were asked to 
estimate how much time they invested 
in mathematics-related professional 
development activities within the past 
year. 

The workshops included in Section A 
at the top of Table 1 focused on the 
new mathematics curriculum for 
students. The Marilyn Bums Institutes 
concentrated on teaching specific 
math topics, while the replacement 
unit workshops (Ms. Burns also 
developed several of these) focused 
on curriculum modules designed to be 
consistent with the mathematics 
reforms, and centered on specific 
topics (such as fractions) or sets of 
topics. 5 Mathematics educators 
devised the high-quality topic- 
centered replacement units to be 
coherent and comprehensive in the 
exploration of mathematical topics, 
and to support teacher and student 
learning. 

The workshops included in Section B 
at the bottom of Table 1 — EQUALS, 
Family Math, and cooperative 
learning — were loosely related to the 
goals of the Mathematics Framework, 
but were not focused directly on the 
student mathematics curriculum. 
EQUALS addresses gender, linguistic, 
class and racial inequalities in math 



Table II 

Teachers’ Learning Opportunities 

California elementary school teachers were asked in which of the following mathematics-related activities they 
participated during the past year and how much total time they spent in each activity (for example, four two-hour 
meetings would equal one day). 



Mathematics-Related Activity 


None 


1 day 
or less 


2 to 6 
days 


1 to 2 
weeks 


More than 
2 weeks 


A. Student Curriculum-Centered Workshops 












Marilyn Burns 


83.2 


9.8 


5.3 


1.3 


.3 


Mathematics Replacement Units 


58.9 


22.7 


14.2 


1.7 


2.5 


B. Special Topics/Issues Workshops 












EQUALS 


96.5 


2.4 


.9 


.2 


0 


Family Math 


81.7 


12.9 


4.3 


.8 


.3 


Cooperative Learning 


54.5 


28.9 


13.7 


1.8 


1.1 



Note: Numbers are percentages of respondents selecting that category, weighted to represent statewide population. Missing data 
assumed to indicate no participation in the mathematics-related activities. 



classes; Family Math helps teachers to 
involve parents in their children’s math 
learning. Cooperative learning 
workshops come in many varieties — 
some, for example, focus on de- 
tracking — but all encourage students to 
learn together. 6 

Two-thirds of the teachers responding 
to the survey reported participating in 
at least one of the five mathematics- 
related professional development 
activities listed in Table 1 , in the year 
just prior to the survey. The breadth 
of these professional development 
opportunities, however, was not 
matched by their depth. 

Table 1 shows that most teachers 
reported spending only nominal 
amounts of time in either type of 
professional development activity. Of 
the teachers who reported attending 
one of the workshops in the past year, 
roughly half indicated they spent one 
day or less than one day in the 
mathematics-related activity. Appro- 




ximately 35 percent reported spending 
between two and six days. A smaller 
fraction of those who attended the 
workshops, and a very small fraction 
of the entire sample, attended 
workshops lasting one week or longer. 

This finding accords with other reports 
about professional development: a 
teacher’s opportunity for professional 
development typically consists of a 
few days each year. 7 Few California 
teachers found rich learning 
opportunities. Most encountered the 
mathematics reforms in conventional 
settings — in a day-long or shorter one- 
shot introduction to a particular 
instructional technique or curriculum. 

Another way of placing these numbers 
in context is to see how they relate to 
teachers’ other opportunities for 
learning about California’s Mathematics 
Framework. In addition to student 
curriculum-centered or special topics/ 
issues specific workshops, teachers 
could have engaged in other activities 

6 



designed to familiarize them with the 
reform, such as participating in reform 
networks, attending meetings of 
mathematics teachers, or serving on 
committees. Table 2 shows that few 
teachers did so. Fewer than six in every 
hundred teachers responding to the 
survey reported attending a national 
mathematics teacher association 
meeting; only twelve or thirteen of 
every hundred teachers reported 
participating in other state or regional 
meetings, teaching local workshops, or 
serving on local curriculum 
committees. Teacher contact with the 
reforms via these leadership activities 
was less frequent than their contact 
through more conventional prof- 
essional development avenues. 

In addition to asking teachers about 
their learning opportunities in the year 
before the survey, we also asked 
whether they had had opportunities to 
learn about the new mathematics 
standards at all in their careers. Sixty- 
five percent of teachers responding to 



the survey reported that they had 
attended school or district workshops 
related to the new mathematics 
standards; 45 percent said they had 
been given time to attend off-site 
workshops or conferences related to 
the standards. Nearly seven out often 
teachers reported participating in one 
or both of these two activities. These 
are only general measures — we have 
no evidence of whether these other 
learning opportunities were long or 
short, were focused on specific 
problems or general principles, or were 
in innovative or conventional formats. 

Tables 1 and 2 could be seen as 
evidence that California policymakers 
and reformers wanted to leverage deep 
changes in mathematics instruction 
with only modest investments. But 
recent research suggests that altering 
the core elements of teaching requires 
extended learning opportunities for 
teachers, generous support from peers 
and mentors, and opportunities to 
practice, reflect, critique, and practice 
again. 8 Such opportunities and support 
were unlikely in the brief professional 
development activities of most 
California teachers. 

But the two tables also offer evidence 
that some reformers undertook a novel 
approach: some teachers’ professional 
development was grounded in the 
improved student curriculum that state 
policymakers had encouraged. Most 
professional development is not 
grounded in student curriculum. By 
comparing the two types of 
mathematics-related activities 
presented in Table 1, we could ask a 
central question: Did teachers who 
attended the student curriculum- 
centered workshops shown in Table 1 
report different kinds of practice than 
those who attended the special topics/ 
issues workshops? 
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Teachers who had opportunities to 
learn about student math curriculum 
reported more of the kind of practice 
supported by the Framework. This 
central finding suggests that the 
content of teachers’ professional 
development can make a difference in 
their classroom practice. Table 3 (see 
page 6) presents evidence of the 
impact that workshop content had on 
teachers’ reports of conventional and 
Framework mathematics instructional 
practices. Here and elsewhere, we 
used regression to sort out the effects 
from different influences on teachers’ 
reported practices, and included three 
control variables — teachers’ famil- 
iarity with reform; their predisposition 
toward mathematics reform; and a 
dummy variable capturing teacher 
attendance at reform workshops in 



years prior to the one measured by 
‘student curriculum’ and ‘special 
topics.’ 9 

As these regressions show, those who 
learned about student math curriculum 
were more likely to report such 
practices as having students discuss 
different ways to solve particular 
problems, having them work on 
extended individual projects, or using 
problems with multiple solutions. The 
average teacher who attended a Marilyn 
Bums or replacement unit workshop 
reported more such practice (nearly 
three-quarters of a standard deviation) 
than the average teacher who did not 
attend those workshops. Moreover, the 
relationship works in both directions. 
Teachers who reported attendance at 
either a Marilyn Bums or replacement 
unit workshop reported fewer 
conventional practices like worksheets 
(about four-tenths of a standard 
deviation) than teachers who did not 
attend them. These student curriculum- 
centered learning opportunities seem 



Table 2 

Teacher Participation in Reform Networks and Leadership Roles 


Network or Leadership Role 


Percent that did 
participate 


Percent that did 
not participate 


Attended a national mathematics 
teacher association meeting 


5.7 


94.5 


Attended a state mathematics teacher 
association meeting, including 
California Mathematics Council 
Affiliates 


12.3 


85.1 


Taught an in-service workshop or 
course in mathematics or mathematics 
teaching 


13.6 


83.5 


Served on a district mathematics 
curriculum committee 


13.7 


84.6 



Note: Teachers were asked to report only for the year prior to the survey. Numbers are 
percentages of respondents selecting that category, weighted to represent statewide 
population. 
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to increase Framework practice and to 
decrease conventional practice. 
Teachers did not just add new 
practices to a conventional core, but 
also changed that core. 

In contrast, workshops not closely tied 
to the student mathematics curriculum 
appear to be unrelated both to the kinds 
of practices California policymakers 
wished to see in schools and to the 
conventional practices (such as 
worksheets and computational tests) 
they were trying to change. 10 We 
suspect this is because the special 
topics/issues workshops, though 
consistent with the state Mathematics 
Framework in some respects, are 
focused less on the mathematics 



teaching practices that are central to 
instruction than on other matters which, 
while relevant to instruction, do not 
chiefly address mathematical content. 
The special topics/issues workshops 
may be useful for some purposes, but 
are likely to be peripheral to 
mathematics teaching — for example, 
incorporating cooperative learning 
groups — rather than changing core 
beliefs and practices about mathematics 
and its teaching. 

The kind of learning that took place in 
the Marilyn Burns and replacement 
unit workshops — learning about the 
mathematics that their students would 
study and learning something about 
teaching and learning that 



mathematics — differs quite sharply 
from most professional development. 
Most teacher professional 
development seems to be generic 
(“classroom management,” for 
example) or peripheral to subject 
matter (such as “using math 
manipulatives”). Generic and 
peripheral professional development 
do not have deep connections to central 
topics in school subjects." In the 
1 980s, there was a modest move away 
from generic pedagogy workshops and 
toward subject-specific workshops, 
such as cooperative learning for 
mathematics, that several observers 
considered an improvement. 12 Our 
survey results, however, suggest that 
teachers’ learning opportunities may 



Table 3 

Associations Between Teachers' Learning Opportunities 
and Teachers' Practice 




Curriculum Only Equations 


Curriculum Plus Time 
Equations 




Conventional 

Practice 


Framework 

Practice 


Conventional 

Practice 


Framework 

Practice 


Intercept 


1.6* 


1.78* 


1.56* 


1.83* 


Student Curriculum-Centered Workshop 


-0.30* 


0.54* 


-0.15** 


0.36* 


Time in Student Curriculum-Centered Workshop 






-0.08* 


0.09* 


Special Topics/Issues Workshop 


0.02 


0.01 


-0.03 


0.04 


Time in Special Topics/Issues Workshop 






0.05 


-0.04 


Previous Framework Learning 


0.02 


0.20* 


0.02 


0.21* 


Affect 


-0.21* 


0.22* 


-0.21* 


0.22* 


Familiarity 


-0.85* 


0.42* 


-0.79* 


0.36** 


R2 (adjusted) 


0.17 


0.22 


0.19 


0.25 



Note: Conventional practice includes having students practice or take tests on computational skills or having students work 
individually on mathematics problems from the textbook or workbook. Framework practice includes having students making 
conjectures and exploring possible methods to solve a mathematics problem, working in small groups on mathematics 
problems or working on projects that take several days. 



* Indicates significance at P<.05 leve 

** Indicates significance at P<. 10 leve 
^nation by Ordinary Least Squarei 
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Table 4 

Associations Between Teachers' Workshop Learning, 
Assessment-Related Learning, and Practice Measures 




Set 1 


Set 2 




Conventional 

Practice 


Framework 

Practice 


Conventional 

Practice 


Framework 

Practice 


Intercept 


1.58* 


1.82* 


1.62* 


1.62* 


Student Curriculum- 
Centered Workshop 


-0.16* 


0.37* 


-0.14 


0.37* 


Time in Student 

Curriculum-Centered 

Workshop 


-0.07* 


0.08* 


-0.06* 


0.07* 


Previous Math 
Workshop 


0.02 


0.21* 


0.06 


0.23* 


Affect 


-0.21* 


0.22* 


-0.17* 


0.11* 


Familiarity 


-0.84* 


0.34** 


-0.61* 


0.35** 


Learned about CLAS 


0.06 


0.002 


0.11“ 


-0.01 


Administered CLAS 


-0.004 


0.14* 


0.06 


-0.02 


CLAS Useful 






-0.14* 


0.21* 


R2 (adjusted) 


0.18 


0.25 


0.21 


0.34 



* Indicates significance at P<. 05 level 
** Indicates significance at P<. 10 level 



need to go one level deeper than mere 
subject specificity. Providing teachers 
with even more concrete, topic- 
specific learning opportunities — in 
fractions, measurement, or ge- 
ometry — appears to help change 
mathematics teaching practices. This 
conjecture is consistent with recent 
research in cognitive psychology 
which holds that learning is domain- 
specific. 13 
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Teachers who spent more time in 
Marilyn Burns and replacement unit 
learning situations reported more 
Framework-related practice and less 
conventional practice. This result 
parallels research on students’ learning 
opportunities: the combination of time 
and content-focus are a potent 
influence on learning. In contrast, time 
spent in special topics/issues 
workshops did not have the same 
payoff in practice. Even large 
investments of time in less content- 
focused workshops were not 
associated with more of the practices 
that reformers want. Again, the effects 
of the special topics/issues workshops 
seem tangential to the central 
classroom issues that are measured by 
our practice scales and on which the 
California mathematics reform 
focused. 

FMooig] FoWo CLAS 
Tests Maid a Limited 
flmiffBtLoeoD©© ©on Praetiee 

Tests are widely believed to be a 
significant influence on teaching. The 
California Learning Assessment 
c - •^"-n (CLAS) was designed partly 




for this purpose. California reformers 
and educators designed an assessment 
that would focus on the new 
conceptions of mathematics and 
mathematical performance advanced 
by the state’s Mathematics 
Framework. 

Two variables were used to measure 
teachers’ responses to CLAS: for 
“Learned about CLAS,” we asked 
teachers if they had participated in any 
activities — such as task development, 
scoring, pilot testing or staff 
development — that provided info- 
rmation about the new assessment; 
and for “Administered CLAS”, we 
asked teachers if they had 
administered CLAS tests. Approx- 



imately one-third of the teachers 
reported that they had learned about 
CLAS; a third also reported they had 
administered CLAS. Yet not every 
teacher who learned about CLAS also 
administered the CLAS tests, and vice 
versa. While there was an association 
between these two variables — 
teachers who administered the CLAS 
tests were more likely to have had an 
opportunity to learn about it — there 
also was enough variance to enable us 
to sort out the effects of learning about 
the test from the effects of actually 
administering it. 

Table 4 shows a statistically significant 
and positive relationship between 
teachers administering CLAS and 
reporting more Framework practice. 



9 



7 



The relationship, however, is quite 
modest: it is not close in size to the 
association between student 
curriculum-centered workshop learning 
and practice. 14 In addition, this 
association between administering 
CLAS and practice does not decrease 
teacher reports of conventional 
practices such as bookwork and 
computational tests. It appears that any 
incentive associated with the 
administration of CLAS only adds new 
practices to existing conventional 
practice. In other words, instead of 
redecorating the whole house, teachers 
supplemented an existing motif with 
more stuff — a result that was also clear 
in our field work. 15 The other CLAS 
variable, whether teachers reported 
they had learned about CLAS, was 
unrelated to teachers’ descriptions of 
their classroom practice in math- 
ematics. In contrast, teachers who spent 
extended time in student curriculum- 
centered workshops reported less 
conventional practice and more 
framework-oriented practice. 

The modest effect of CLAS 
administration might disappoint 
supporters of assessment-based 
reform because it suggests that the 
incentives associated with testing 
alone are not great. It should be noted, 
however, that the CLAS lasted only 
two years and published results only 
at the school level, which may have 
been insufficient for incentive effects 
to develop. 

FiMiiinig) Fow®° CLAS as 
Lesunroflinig ©pp®irftOTi% 
Mad Wota ©on 
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Not all teachers who reported 
administering the CLAS either agreed 
with the test’s orientation or modified 
their teaching to fit the test. Many 
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teachers who administered the test liked 
it and used it as a learning opportunity, 
but others did not. Still others did not 
administer the test, but reported they 
had learned new kinds of mathematics 
and instructional practices in CLAS 
workshops. The effects of testing on 
teaching practice were seen among 
these teachers who considered 
themselves to be learners about and 
sympathizers with the CLAS, as 
captured by the CLAS useful variable 
in Table 4, Set 2. This relationship works 
both ways: teachers in this group 
reported more framework practices and 
fewer conventional practices. 

Instead of compelling teachers to teach 
the mathematics to be tested, the 
CLAS seems to have provided 
teachers with occasions to observe, 
think about, and revise mathematics 
instruction. Some teachers seized the 
occasion while others ignored it. One 
teacher, interviewed in a related study, 
reported: 

“...the CLA S test. ...It was a shock 
to me. They [students] really did 
fall apart. It was like, ‘Oh! What 
do Ido? 'And I realized, I need to 
look at mathematics differently. 
You know, I really was doing it the 
way I had been taught so many 
years before. I mean, it was so 
dated. And I began last year, 
because of the CLAS test the year 
before, looking to see what other 
kinds of things were available. ” 16 

This teacher’s learning (“...looking to 
see what other kinds of things were 
available”) and her efforts to change 
her practice were associated with the 
incentive created when she noticed that 
her students “...really did fall apart” 
when taking the new test. Her 
students’ weak test performance 
inspired her to help them do better, 
even before she saw any test results. 



Thus, California’s use of curriculum- 
aligned assessment did not act as a 
simple lever to enforce instructional 
change — perhaps because of its weak 
external accountability mechanism — 
but rather as an incentive and resource 
for teachers to learn. Further, its effects 
were selective: many teachers who 
came in contact with the CLAS were 
stimulated to reevaluate their math 
instruction, but the test was a resource 
or incentive only to those who 
perceived it as such. State reformers 
shrewdly provided numerous 
opportunities for those teachers to 
learn about the assessment and 
mathematics it contained, including 
involving a small group of teachers in 
piloting and scoring the test and 
encouraging this small group to offer 
workshops to the wider teacher 
population. Observers of math reform 
reported that these learning 
opportunities consisted of viewing the 
types of mathematics problems 
advanced by the Frameworks; 
examining student responses to those 
problems closely; and scoring student 
work based on state rubrics. In other 
words, participating teachers were 
exposed to elements of a curriculum 
on improved math teaching. 

Fnimdlmg Bm: CeirftaoBU 
Teasteir Learning 
Opp®rtam)Dto®s A1fff@©ti@dl 
SftLodemrfi Aelhilleveinnieinitt 

Our results suggest that links between 
instructional policy and classroom 
practice may be expected when 
teachers’ learning opportunities are: 

• grounded in the curriculum that 
students study; 

• connected to several elements of 
instruction (for example, not only 
curriculum but also assessment); 
and 

• extended in time. 



Such learning opportunities are not 
common in American education. 
Professional development has rarely 
been grounded in the academic content 
of schooling or in knowledge about 
student performance. This may 
explain why so few studies of 
professional development report 
connections to teachers’ practice, and 
why so many studies of instructional 
policy report weak implementation; 
teachers’ work as learners was not tied 
to the academic content of their 
students’ work. 

A critical question remains: Did 
students of teachers who received 
professional development centered 
on student curriculum score higher 
on the state mathematics assess- 
ments? 

To explore this question, we merged 
fourth grade mathematics scores from 
the 1994 administration of the 
California Learning Assessment 
System (CLAS) with the school files 
in our data set. The CLAS included a 
good deal of performance-based 
assessment. To do well, students had 
to answer adequately a combination of 
open-ended and multiple-choice items 
that were designed to tap their 
understanding of mathematical 
problems and procedures. State 
scorers assigned scores from Level- 1 
to Level-6 based on student 
proficiency levels; school scores were 
reported as “percent of students 
scoring Level-1” and so on. We 
created an average of these for each 
school to represent our CLAS 
dependent variable, with the higher 
school scores representing a more 
proficient student body. 17 

Because the California Department of 
Education reported only school-level 
scores, we had to compute school 
averages of all independent variables, 
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including teachers’ reports of practice 
and learning opportunities. The survey 
sampled only four or fewer teachers 
per school, so the averages provided 
only a crude estimate of our 
independent measures. Therefore, 
these measures of school engagement 
with reform are error-filled, that is, 
they are likely to bias the investigation 
against finding significant results, 
because random noise in equations 
diminishes the effects on affected 
variables. 

We created three additional variables 
for each school in the sample: a 
measure of student socio-economic 
status; a measure of school conditions; 
and a measure of replacement unit 
use. 18 In addition to these three, we 
continued to use the variables (all 
averaged for schools) that mark other 
potential connections between policy 
and practice, including time spent in 
certain workshops, 19 our control for 
teachers’ previous Framework learning 
experiences, 20 teachers’ reports of 
Framework practice, and a CLAS 
Learning Opportunities measure 
(indicating teacher-reported attend- 
ance at a state or district-sponsored 
opportunity to learn about the new 
mathematics assessment). 

The central issue in this analysis is 
whether the evidence supports our 
model of relationships between policy 
and performance, and whether this 
relationship is mediated by teachers’ 
practice. Equation 1 in Table 5 shows a 
modest relationship on the second count: 
schools in which teachers report 
classroom practice that is more oriented 
to the Mathematics Framework have 
higher average student scores on the 
fourth grade 1 994 CLAS, controlling for 
demographic characteristics of schools. 
No such relationship was found 
between schools high on our 
conventional practice scale and student 




achievement scores. This provides 
evidence that teachers’ practice links 
the goals and results of state policy: 
students benefited from having teachers 
whose work was more closely tied to 
state instructional goals. 

Based on this analysis, we would 
expect a modest relationship between 
many of our policy and learning 
indicators and student outcomes as 
well because the policies encouraged 
framework practice. This is true for 
our measure of teacher attendance at 
student curriculum-centered work- 
shops: teachers who attended these 
workshops have students who 
performed better on the mathematics 
CLAS, as is evident in Equation 2 in 
Table 5. 

A more important issue may be the 
effect teacher learning in the special 
topics/issues workshops had on 
student achievement. We saw earlier 
that this variable contributed little to 
explaining differences among 
teachers’ Framework or conventional 
mathematics practice. Hence, any 
effect we might find on student 
achievement would be through 
pathways not detected by those scales, 
such as increasing teacher knowledge, 
improving equity within classrooms, 
or helping teachers better understand 
student learning. But we found no such 
effect of the special topics/issues 
workshops on student achievement. 
This is an important result: whatever 
improvements these workshops may 
bring to California’s classrooms, they 
do not affect what many see as the 
bottom line of schooling — student 
performance. 

The third component of the policy mix, 
the use of replacement units, shows a 
positive relationship to student 
achievement. Equation 3 in Table 5 
shows that schools in which teachers 
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Table 5 

Associations Between Teachers' Practice, Their Learning Opportunities and 1994 Student Mathematics 
Scores on the California Learning Assessment System (CLAS) 




Equation-1 

CLAS 


Equation-2 

CLAS 


Equation-3 

CLAS 


Equation-4 

CLAS 


Equation-5 

CLAS 


Intercept 


2.14* 


2.65* 


2.67* 


2.66* 


2.57* 


Percent of Students Eligible for Free Lunch 


-1.17* 


-1.23* 


-1.21* 


-1.24* 


-1.21* 


School Conditions 


0.19* 


0.17* 


0.17* 


0.18* 


0.17* 


Framework Practice 


0.17* 










Traditional Practice 


-0.00 










Student Curriculum-Centered 
Workshops - Time 




0.065* 






0.041** 


Special Topics/Issues Workshops - Time 




0.03 








Replacement Units - Average Number Used 






0.11* 




0.11* 


Learned About CLAS 








0.21* 


0.15** 


Past Framework Learning 




0.11 






0.14 


R2 (adjusted) 


0.60 


0.60 


0.61 


0.60 


0.62 



Note: All survey-based measures are averages from the teachers within a school who responded to the survey. 

* Indicates significance at P<.05 level. 

** Indicates significance at P<. 15 level. 



reported that they each used one 
replacement unit had higher student test 
scores (by an average of approximately 
one quarter of a standard deviation) than 
schools in which no teachers reported 
use of replacement units. 

Finally, we come to the effect on 
achievement associated with teacher 
learning about the California Learning 
Assessment System (CLAS). 21 The 
coefficient on Learned about CLAS 
(Equation 4 in Table 5) suggests a clear 
effect: when comparing student 
achievement scores, schools where all 
teachers learned about the CLAS had 
higher student test scores (roughly one- 
quarter of a standard deviation higher) 
than schools where no teachers learned 
about CLAS. 
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We might ask whether one of these 
policy initiatives is dominant in improving 
student scores, or whether they work 
independently, each exerting its own 
influence. It appears, based on Equation 
5 in Table 5, that they work in- 
dependently: each continues to 
contribute to higher student test scores, 
even when controlling for the others. 
In other words, California’s in- 
structional policies have cumulative 
effects on student learning. 

Our practice and policy measures 
related positively to student 
achievement. Although this inter- 
pretation is based on aggregate data, it 
is difficult to think of any other 
reasonable inference save that 
teachers’ learning opportunities can 
pay off in their students’ performance 
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if the conditions of teachers’ learning 
summarized in our model — grounded in 
student curriculum, connected to 
several elements of instruction, and 
extended in time — are satisfied. This 
suggests that well-planned state efforts 
to improve instruction can successfully 
influence not only teaching but also 
student learning. 

CobksOmsoobi 

The 1994 survey of California 
elementary school teachers tells us that 
teachers’ opportunities to learn about 
reform do affect their practice. When 
these learning opportunities are 
focused on student curriculum that is 
designed to be consistent with the 
reforms, and that their students study, 
teachers report practice that is 



significantly closer to the aims of the 
instructional policy. In these situations, 
there is a consistent relationship among 
the following: the professional 
curriculum of reform; the purposes of 
policy, assessment and teachers’ 
knowledge of assessment; and the 
student curriculum. Finally, when the 
assessment of students’ performance 
is consistent with the professional and 
the student curriculum, teachers’ 
learning opportunities pay off in 
students’ math performance. These 
results confirm the analytic usefulness 
of an instructional model of 
instructional policy, and suggest the 
potent role that the education of 
professionals can play in improving 
public education. 

If our analysis is correct, teaching 
practice and student performance are 
likely to improve when educational 
improvement is focused on learning 
and teaching academic content, and 
when curriculum for improving 
teaching overlaps with curriculum and 
assessment for students. Under such 
circumstances, educational policy is an 
instrument for improving teaching and 
learning. Policies that do not meet 
these conditions — for example, new 
assessments or curricula that are not 
accompanied by adequate learning 
opportunities for teachers, or 
professional development that is not 
grounded in academic content — are 
less likely to have constructive 
effects . 22 

These points have important bearing 
for the professional development 
system, or perhaps, non-system. 
Professional development that is 
fragmented, that is not focused on 
curriculum for students, and that does 
not afford teachers consequential 
learning opportunities cannot be 
expected to be a constructive agent of 
state or local instructional policy. Yet, 
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professional development in the 
United States today. Teachers typically 
engage in a variety of short-term 
activities that fulfill state or local 
requirements for professional learning, 
but which are rarely deeply rooted in 
the school curriculum or in thoughtful 
plans to improve teaching and learning. 
Our study confirms that picture, and 
shows further that neither teachers’ 
practice nor students’ achievement 
was changed by most of the 
professional development experienced 
by most California teachers. Still, large 
amounts of money are spent every year 
on just such activities . 23 Our results, 
therefore, present a challenge to those 
who make policy for and practice 
professional development: Can they 
design policies, programs, and 
requirements that focus far more 
closely on improved teaching for 
improved student learning? 

Our analysis appears to confirm 
arguments in favor of standards-based 
reform in that it broadly supports an 
integrated approach to school 
improvement — an approach that leads 
to the creation of better curriculum for 
students, that makes suitable provision 
for teachers to learn that curriculum, 
that focuses teaching on learning, and 
that thoughtfully links curriculum and 
assessment to teaching. Some 
examples of standards-based reform 
meet these criteria while others do not. 
Some other approaches to school 
improvement that do not fly the banner 
of standards-based reform also meet 
these criteria . 24 

The story we have related is not a story 
in which the efforts of state agencies 
carried the day. Rather, it is a story in 
which the related actions of 
government and professional 
organizations were crucial. California 
state agencies played a key role in 
framing a set of ideas about improved 
math teaching and learning, in 
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supporting those ideas, and in 
changing some state education 
requirements to be more consistent 
with the ideas. The state alone, 
however, did not have the educational 
resources to frame those ideas. This 
required extensive help from 
educators — through agencies as 
diverse as the National Council of 
Teachers of Mathematics and its 
California affiliate, home-office 
curriculum developers, and university 
schools and departments, among 
others. Changes in teaching practice 
depended on professional as much as 
state action . 25 

Working together, this diverse set of 
agents was able to create somewhat 
more coherent relationships among 
teachers’ learning opportunities, their 
practice, school curriculum and 
assessments, and student achievement, 
than are typical in the U.S. And, our 
evidence shows that California 
reformers, after years of hard work, 
were able to do so for only fifteen to 
twenty percent of the state’s teachers. 
The obstacles fit with what we know 
about fragmentation in the U.S. public 
education system: it is more nearly a 
non-system, a sprawling organization 
that makes it difficult to organize 
coherent and concerted action within 
even a single modest-sized school 
district, let alone an entire state . 26 

These findings also square with recent 
research on teachers’ learning and 
change: certain sorts of learning 
opportunities do seem to alter 
teachers’ practice and student learning, 
but change typically occurs slowly and 
partially. The teachers who took most 
advantage of new learning 
opportunities blended new elements 
into their practice while reducing their 
reliance on, but not abandoning, some 
older practices. 
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A final note: All of our findings rest on 
non-experimental evidence which is not 
conclusive. The relationships that we 
have reported should be investigated 
with a larger population of schools and 
teachers, in a longitudinal format, so 
that more robust causal attributions 
might be probed, and more precise 
measures employed. We expect to 
organize such a study. 

But the results that we report here do 
not come from left field: they seem 
reasonably robust, and are consistent 
with several related lines of recent 
research. 27 Although we think better 
research on these issues is essential, 
we would be surprised if the direction 
of the effects we have found, and our 
model of causation, did not stand up 
in a more powerful design. Therefore, 
we think it would be wise for 
policymakers and practitioners to 
ground teachers’ professional 
education more solidly in deeper 
knowledge of the student curriculum. 
When designing new curricula and 
assessments, we think it would be wise 
to make more adequate provision so 
teachers could learn about and from 
the new curricula and assessments. 
And we think it would be wise to offer 
teachers more opportunities to relate 
assessments to curricula, and both to 
pedagogy. 
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